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Abstract 

The  federal  government  is  becoming  increasingly  “corporate”  and,  consequently, 
business-like  operations  are  now  more  prevalent  than  ever.  Part  of  the  mandate  to  “act  like  a 
business”  is  a  need  to  develop  strategic  plans  and  goals,  which  require  metrics  to  quantify  what 
is  to  be  attained.  In  this  environment,  an  effective  performance-management  program  is 
essential  to  success. 
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The  ASN  (RD&A)  has  updated  its  strategic  plan  in  support  of  the  DoD’s  efforts  to 
transform  the  military  based  on  these  environmental  changes.  Its  efforts  to  implement  a  revised 
performance-management  program  in  support  of  its  revised  plan  offer  potential  insights  into 
performance  measurement  in  defense  acquisitions. 

Introduction 

The  federal  government  is  becoming  increasingly  “corporate”  with  ever  more  focus  on 
issues  such  as  competitive  sourcing  (A-76),  President’s  Management  Agenda  (PMA),  OMB  300, 
the  Performance  Assessment  Rating  Tool  (PART),  the  Government  Performance  and  Results 
Act  (GPRA),  and  Management  Initiative  Decision  (MID)  901,  to  name  a  few.  And,  with  its  focus 
on  transformation  under  the  Bush  Administration,  the  Department  of  Defense  (DoD)  is  part  of 
that  trend.  Under  Secretary  Donald  Rumsfeld,  the  DoD  has  reorganized  to  become  more  like  a 
business,  with  management  teams,  a  top-down  strategy,  and  a  focus  on  performance-based 
management.  Consequently,  business-like  operations  are  becoming  more  prevalent  in  the 
Department,  and  the  DoD  (like  other  agencies)  is  incorporating  to  a  greater  degree 
management  artifacts  like  cost  analysis,  cost  accounting,  trade-off  analysis,  and  performance 
measurement  into  daily  operations.  As  the  Government  Accountability  Office  (GAO)  states  in  its 
report  on  performance  budgeting:  “Performance  language  and  tools  have  become  part  of  the 
culture  of  governance”  (GAO,  2005,  22). 

Part  of  the  mandate  to  “act  like  a  business”  includes  a  need  to  develop  strategic  plans, 
as  well  as  objective  goals  around  those  plans.  The  goals,  then,  require  metrics  to  quantify  what 
is  to  be  attained,  followed  by  processes  to  track  and  report  on  progress  against  those  goals. 
With  its  transformation,  the  DoD,  like  many  organizations,  needs  to  make  difficult  management 
decisions — trade-offs  among  several  worthwhile  projects — and  it  requires  strong,  meaningful, 
verifiable  information  to  do  so.  That  kind  of  information  comes  only  from  a  good  performance- 
metrics  program. 

In  support  of  the  DoD’s  decision  to  transform  the  military,  and  understanding  the  highly 
dynamic  acquisition  environment,  John  J.  Young,  Jr.,  Assistant  Secretary  of  the  Navy  for 
Research,  Development  and  Acquisition  (ASN  (RD&A))  recognized  the  need  to  revise  and 
update  the  Naval  Research  and  Acquisition  Team  1999-2004  Strategic  Plan.  Mr.  Young 
identified  two  key  needs  within  the  community  that  the  updated  plan  had  to  address:  (1 )  to 
develop  systems  flexible  enough  to  respond  to  the  many  different  challenges  the  organization 
could  face;  and,  (2)  to  create  an  organization  that  could  reinvent  itself  on  an  ongoing  basis,  not 
only  in  response  to  specific  threats. 

Mr.  Young’s  revised  plan,  the  Blueprint  for  the  Future,  starts  with  a  vision  statement: 

Build  a  strategic  capability  to  strike  anyone,  anywhere,  anytime. 

The  vision  statement  is  supported  by  three  vision  concepts: 

1 .  Strategic  Interests:  must  think  globally 

2.  Strategic  Awareness:  must  be  able  to  collect,  analyze,  and  communicate 
information 

3.  Strategic  Resilience:  must  be  resilient/innovative 
Figure  1  illustrates  ASN  (RD&A)’s  vision  concepts. 
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Figure  1:  ASN  (RD&A)’s  Pyramid  of  Strategic 


I!.  Strategic  Resilience  ill.  Strategic  Awareness 
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The  vision,  in  turn,  is  supported  by  three  principles,  under  which  Mr.  Young  outlines 
high-level  organizational  goals  and  initiatives/projects  for  fulfilling  the  strategic  plan: 

Principle  Number  1 :  The  Naval  Acquisition  Team  must  think  like  a  business  and 
run  a  tight  ship. 

Specific  Goals:  We  will  work  with  industry  on  a  business-to-business  basis 
and  measure  our  organic  businesses  against  the  best  industry  benchmarks. 

Sample  Goals: 

•  Every  program  will  seek  to  continuously  cut  government  and  industry  cost. 

•  Each  SYSCOM  Commander,  PEO,  and  PM  should  ensure  that  at  least  5  lean 
events  are  held  in  each  depot  or  industrial  activity — government  or  industry. 

Initiatives/Projects:  The  Naval  Acquisition  Team  will  deliver  to  budgets  and 
schedules  we  define. 

Sample  Initiatives/Projects: 

•  Deliver  LPD-1 7,  SSN-775,  and  the  USS  EISENHOWER  to  the  Fleet. 

•  Complete  contracts  for  DDX  lead  ship  construction,  SHA(R),  and  MPF(F). 
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Principle  Number  2:  The  Naval  Acquisition  Team  must  innovate  and  collaborate 
to  deliver  effective,  affordable  weapons  for  Sailors  and  Marines. 

Specific  Goals:  We  must  change  how  we  do  business  in  both  major  and 
incremental  ways  to  deliver  resilient  strategic  capability  at  the  lowest  possible  cost. 

Sample  Goals: 

•  Define  the  path  to  ForceNet  by  collaborating  with  both  the  operational  and 
requirements  communities  and  with  our  allies 

•  Define  Sea  Basing  concepts  and  work  with  the  Army  and  Air  Force  to  jointly 
enable  these  programs 

Initiatives/Projects:  The  Naval  Acquisition  team  must  use  incentives  and 
metrics  to  hold  industry  and  ourselves  accountable. 

Sample  Initiatives/Projects: 

•  Contracts  are  the  key  management  tools  for  program  managers,  and  DASNs, 
PMs,  and  PEOs  should  ensure  that  an  appropriate  profit/incentive  strategy 
structure  is  included  in  all  new  contracts. 

•  DASNs,  PEOs  and  PMs  will  define  realistic  budgets  and  schedules  to  deliver 
appropriate  capabilities  and  will  be  prepared  to  be  held  accountable  to  those 
plans. 

Principle  Number  3:  The  Naval  Acquisition  Team  will  operate  as  a 
neighborhood  to  jointly  integrate  systems  and  develop  people. 

Specific  Goals:  The  Naval  Acquisition  Team  must,  as  individuals,  take 
responsibility  for  growth  and  enhancement  of  our  neighborhood. 

Sample  Goals: 

•  Every  person  in  the  neighborhood  should  daily  seek  to  change  things  to  more 
effectively  and  efficiently  deliver  value  for  the  warfighter  and  taxpayer. 

•  Managers  will  ensure  that  our  system  rewards  unselfish  collaboration  and 
innovation. 

Initiatives/Projects:  We  will  create  an  Enterprise  Culture  and  achieve 
Operational  Excellence:  $1  billion  in  real  improvements. 

Sample  Initiatives/Projects: 

•  We  need  to  think  and  act  like  a  fleet-footed  business — instead  of  a  big 
bureaucracy  that  moves  at  glacial  speed — if  we  are  to  realize  our  vision  of  strike 
anyone,  anywhere,  anytime. 

•  We  need  to  develop  clear  acquisition  strategies  and  goals  in  a  collaborative 
manner  with  our  customers  in  the  Navy  and  supply-chain  partners  in  the  Defense 
Industry. 

UPDATING  ASN  (RD&A)’S  BSC 

Given  the  effectiveness  of  the  BSC  model  within  other  like  organizations,  it  is  an  optional 
model  for  ASN  (RD&A)’s  leaders  (working  under  Mr.  Young’s  direction)  to  implement  the 
Blueprint.  The  original  four  perspectives  of  the  ASN  (RD&A)’s  BSC  are  still  valid: 

1 .  Warfighter:  How  do  customers  see  us? 

2.  Internal  Process:  At  what  must  we  excel? 
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3.  Growth  &  Learning:  Do  we  continue  to  improve  and  create  value? 

4.  Financial:  Do  we  provide  the  best  capability  for  the  warfighter? 

Figure  2  shows  the  ASN  (RD&A)’s  four  BSC  perspectives. 

As  part  of  the  initiative  to  update  the  organization’s  BSC  to  support  the  Blueprint,  ASN 


Figure  2:  Four  Perspectives  of  the  ASN 


How  do  our  customers  see  us? 


Do  we  continue  to 
improve  and  create  value? 


(RD&A)  managers  should  review  and  update  the  Balanced  Scorecard  (BSC)  with  a  focus  on 
metrics  management,  keeping  in  mind  the  need  for: 

•  Baseline  review:  establishing  processes  necessary  to  determine  current  status, 
focusing  on  the  basis  for  performance-data  analysis. 

•  Performance-data  collection  and  analysis:  placing  greater  emphasis  on 

activity-based  cost 

♦  operational  and  support  (O&S)  cost  accounting 
economic  trade-off  analysis. 

•  Performance  measurement:  incorporating  metrics  into  BSC  and  the  strategic  plan. 

•  Flexibility:  incorporating  change  mechanisms  to  accommodate  the  new 
environment. 

•  Community  involvement:  involving  stakeholders  inside  and  outside  ASN  (RD&A)  to 
offer  input  and  support. 

•  Institutional  commitment:  obtain  institutional  commitment  at  all  levels. 
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ASN  (RD&A)’S  BSC:  INFLUENCING  FACTORS 

As  with  most  organizations,  ASN  (RD&A)  does  not  work  in  a  vacuum.  It  has  many 
influencing  factors  to  consider  as  it  develops  and  implements  its  BSC  model. 

Challenges 

ASN  (RD&A)  leaders  face  multiple  challenges — some  common  to  other  organizations, 
some  unique  to  their  own — that  could  impede  efforts  to  effectively  implement  its  BSC: 

•  Proliferation  of  agendas:  Many  agendas  affect  the  organization — individual  staff, 
the  many  areas  of  ASN  (RD&A)  itself,  DoN,  DoD,  government-wide,  Congressional, 
etc. 

•  Changing  control:  An  ASN  (RD&A)  BSC  must  meet  current  needs,  but  be  flexible 
enough,  and  sensible  enough,  to  succeed  through  regular  control  changes,  such  as 
Congressional  and  Presidential  elections. 

•  Unforeseen/uncontrolled  “environmental”  changes:  The  ASN  (RD&A)  is  greatly 
affected  by  environmental  issues  over  which  it  may  have  limited  control,  such  as  the 
changes  brought  on  by  the  end  of  the  Cold  War  or  the  events  of  9/11/01 . 

•  Overlapping  roles  and  responsibilities:  Within  the  Navy,  overlapping 
responsibilities  in  areas  such  as  requirements  and  budgets  impact  ASN  (RD&A)’s 
strategic  plans  and  metrics  management. 

•  Changing  focus:  The  short  timeframe  of  civilian  political  appointees  and  military 
leadership  can  dilute  institutional  energy  for  some  projects. 

•  Stakeholder  consensus:  Lack  of  consensus  by  stakeholders  on  the  selection  of 
measures  reported  often  scuttles  an  otherwise  well-developed  plan. 

•  Data  reliability:  Any  program  that  relies  on  data  faces  concerns  about  the  reliability 
of  the  data,  and  ASN  (RD&A)’s  program  must  alleviate  these  concerns — both  by 
ensuring  it  has  the  technical  capability  to  collect  and  analyze  the  data,  as  well  as  by 
gaining  stakeholder  buy-in  on  the  method  of  collection. 

•  Data  volume:  Large  quantities  of  data  in  ASN  (RD&A)  can  limit  their  usefulness  by 
making  it  difficult  for  decision  makers  to  ascertain  the  most  relevant  information. 

Existing  Performance-Management  Initiatives 

At  the  same  time,  ASN  (RD&A)’s  BSC  must  integrate — or  at  least  consider — many 
existing  performance-measurement  initiatives: 

•  GPRA:  The  Government  Performance  and  Results  Act  of  1993  requires  federally 
funded  agencies  develop  and  implement  an  accountability  system  based  on 
performance  measurement,  including  setting  goals  and  objectives  and  measuring 
progress  toward  achieving  them. 

•  CFOA:  The  Chief  Financial  Officers  (CFO)  Act  of  1990  requires  DoN  to  provide 
auditable  financial  statements  that  link  performance  measures  and  financial 
information. 

•  PMA:  The  President’s  Management  Agenda  (PMA)  is  a  coordinated  strategy  to 
reform  federal  management  and  improve  program  performance.  It  targets  five 
government-wide  initiatives  for  every  department  and  agency:  Strategic  Management 
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of  Human  Capital,  Competitive  Sourcing,  Improved  Financial  Performance, 
Expanded  e-Government,  Budget  and  Performance  Integration. 

•  PART:  PMA’s  Performance  Assessment  Rating  Tool  (PART)  is  one  of  five  cross¬ 
cutting  initiatives  in  the  PMA.  PART  uses  a  set  of  questions,  the  answers  to  which 
are  translated  into  a  numeric  score  designed  to  assess  program  performance  in  four 
areas:  Purpose,  Strategic  Planning,  Program  Management,  and  Program  Results. 

•  CM:  PMA’s  Common  Measures  section  is  also  part  of  the  PMA’s  Budget  and 
Performance  integration. 

•  SEC  Metrics:  SECDEF  established  the  Senior  Executive  Committee  to  lead  the 
revitalization  process.  The  Deputy  Secretary  leads  the  SEC,  which  consists  of  the 
Service  Secretaries  and  USD  (AT&L).  The  SEC  directed  the  formation  of  the  DoD 
Metrics  Working  Group.  The  executive  secretary  of  the  SEC  leads  the  working 
group,  which  consists  of  members  from  OSD/PA&E,  P7R,  AT&L,  and 
representatives  from  each  of  the  Services  and  the  Joint  Staff.  The  DoD  Metrics 
Working  Group’s  task  is  to  develop  metrics  strawmen  with  proposed  measures  for 
each  of  the  DoD  Risk  Management  Areas. 

•  Management  Initiative  Decision  (MID)  901:  Establishes  performance  outcomes 
and  tracks  performance  results  for  the  DoD  in  consolidating  the  management  goals 
in  the  PMA;  also  utilizes  Quadrennial  Defense  Review  (QDR)  performance  goals 
under  a  balanced  scorecard  and  designates  the  metrics  SECDEF  will  use  to  track 
associated  performance  results.  The  SEC  is  coordinating  with  the  services  in 
building  metrics  and  a  strategic  plan  to  a  framework  outlined  in  MID  901. 

•  AT&L  BSC:  AT&L  already  had  initiated  development  of  a  BSC  to  address  its 
priorities  in  response  to  QDR. 

•  SECNAV:  SECNAV  has  implemented  its  own  initiative  to  create  effective 
performance  measures  within  DoN. 

•  ACAT  Program  Metrics:  These  metrics  address  program  effectiveness  for 
Acquisition  Category  I  &  II  programs. 

•  Acquisition  Economics:  Economic  Order  Quantity. 

•  RDA  Strategic  Plan:  Finally,  of  course,  ASN  (RD&A)  had  to  address  its  own 
performance  measures. 

Sources  of  Strategic  Guidance 

In  addition  to  its  common  and  unique  challenges  and  existing  performance  metrics,  ASN 
(RD&A)’s  efforts  are  further  impacted  by  the  abundance  of  strategic  guidance — some  of  which 
is  found  in  the  same  programs  that  outline  performance  metrics,  and  others  that  offer  only 
strategic  direction: 

•  National  Security  Strategy 

•  PMA 

•  Quadrennial  Defense  Review  (QDR) 

•  Defense  Planning  Guidance 

•  Naval  Power  21 

•  Government  Performance  and  Results  Act  (GPRA) 
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•  Annual  Defense  Report 

•  Department  of  the  Navy  Posture  Statements 

•  Department  of  the  Navy  Playbook 

These  challenges,  existing  metrics,  and  strategic  directions  create  sometimes 
inconsistent  and/or  overlapping  needs,  data  needs  that  could  overwhelm  operations,  and  a  lack- 
of-context  to  ensure  that  different  initiatives  will  be  effective. 


Figure  3  shows  the  impacts  on  ASN  (RD&A)’s  strategic-planning  effort. 

Figure  3:  ASN  (RD&A)  BSC  Influencing 


Government-Wide 

Initiatives/Issues 

•  GPRA 

•  PMA 

•  PART 

•  CM 

•  CFOA 

•  ACAT  Program  Metrics 

•  Acquisition  Economics 

•  Changing  Control 


DoN  Initiatives/Issues 

•  Naval  Power  21 

•  DoN  Strategic  Plan/BSC 

•  DoN  Posture  Statements 

•  DoN  Playbook 

•  Navy  AT&L  Goals  and 
Priorities 

•  RDA  Strategic  Plan 


Operational  Issues 

•  Proliferation  of  Agendas 

•  Unclear  Lines  of 
Authority 

•  Stakeholder  Consensus 

•  Data  Reliability 

•  Data  Volume 


DoD  Initiatives/Issues 

•  SEC  Metrics 

•  QDR 

•  Defense  Planning  Guidance 

•  DoD  Transformation  Goals 

•  DoD  Strategic  Plan/BSC 

•  MID  901 

•  National  Security  Strategy 

•  Annual  Defense  Report 
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IMPLEMENTING  THE  BLUEPRINT  THROUGH  A  BSC  MODEL 

One  way  to  implement  Mr.  Young’s  vision  is  through  a  balanced  scorecard  approach. 
The  following  information  is  relevant  if  this  model  is  the  chosen  path. 

Issues  to  Address 

In  order  for  DASNs,  PEOs,  and  SYSCOM  Commanders  to  effectively  utilize  Mr.  Young’s 
vision  using  a  BSC  approach,  they  will  need  to  address  the  following  issues. 

•  Focus  metrics:  Effective  performance  management  efforts  must  expand  beyond 
cost,  schedule,  and  risk.  If  they  do  not,  they  run  the  risk  of  concentrating  on  activities 
that  are  not  necessarily  those  most  important  to  the  support  of  the  community’s 
strategic  goals. 

•  Think  longer  term:  Part  of  Mr.  Young’s  strategy  is  to  plan  for  a  longer  time  horizon. 

•  Orient  metrics  to  outcomes:  The  metrics  being  used,  and,  therefore,  the 
performance  being  measured,  need  to  focus  on  outcomes — not  activities. 

•  Create  a  line-of-site:  Each  PEO,  DASN,  and  SYSCOM  Commander  must  have  a 
scorecard  to  create  a  line-of-site  throughout  the  organization  that  demonstrates  how 
the  work  of  each  employee  supports  the  organization’s  overall  goal. 

•  Create  accountability:  Implementing  metrics  throughout  the  organization  drives 
accountability. 

•  Select  SMART  metrics:  Metrics  need  to  be  SMART,  as  well  as  consistent, 
understood,  accepted,  and  evaluated  (at  least  on  a  quarterly  basis). 

•  Align  metrics  with  DoN,  DoD,  and  government-wide  initiatives:  Metrics  need  to 
demonstrate  a  relationship  to  the  broader  initiatives  of  the  organizations  of  which 
ASN  (RD&A)  is  a  part.  ASN  (RD&A)’s  performance-management  program  must 
support  its  own  goals  first-and-foremost,  but  the  absence  of  alignment  to  outside 
performance  metrics  simply  creates  an  additional  layer  of  collection  and  analysis. 

Putting  it  Together:  Start  Large  and  Work  Down  to  the  Detail 

DoN  Strategic  Context 

The  DoN  provides  a  context  for  strategy  and  metrics  development  and  implementation 
starting  at  the  top  and  working  down  through  the  organization  at  ever-increasing  levels  of  detail. 
Figure  4  depicts  the  Navy’s  strategic  context. 
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Figure  4:  Navy’s  Strategic  Context 


DoN  Combined  Metrics  Effort 

The  Navy’s  vision  includes  a  single  coherent  effort  to  develop  performance  metrics  that 
address  both  long-range  goals  for  current  capabilities  as  well  as  for  transformation.  The  vision 
calls  for  its  units  to  proceed  from  a  common  Navy  vision  and  strategy,  and  then  to  consolidate 
efforts  to  develop  metrics  that: 

1 .  make  sense  for  the  Navy 

2.  are  useful  to  decision-makers  and  managers 

3.  are  effective  for  managing  performance  and  driving  change 
Figure  5  demonstrates  how  the  DoN’s  metrics  efforts  interact. 
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Figure  5:  Combined  DoN  Metrics 


Span  of  Control 

In  developing  its  performance  measures  from  top  to  bottom,  managers  need  to  consider 
the  timing  and  impact  of  each  stratified  level  of  performance  measurement.  Clearly,  as  spans  of 
control  decrease,  impacts  decrease.  Yet,  at  the  same  time,  effort  needed  to  implement  plans 
also  decreases  as  spans  of  control  decrease.  Important  points  to  remember  in  thinking  about 
stratified  performance  measures  are  (a)  the  measures  must  work  like  a  pyramid,  each  level 
supporting  the  level  above  it,  and  (b)  the  impacts  of  decisions  will  decrease  as  spans  of  control 
decrease.  Figure  6  illustrates  how  span  of  control  relates  to  level  of  impact. 

Figure  6:  Span  of  Control  Relative  to 


*Time  Required  to  evaluate  implications 
NOTE:  Strategies  guide  activities  at  all  levels 
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ASN  (RD&A)  Integrated  BSC 

With  a  full  understanding  of  the  government-wide  initiatives,  of  DoD’s  strategic  plan,  of 
DoN’s  strategic  plan,  of  the  Navy’s  vision,  strategic  context,  and  combined  metrics  effort,  of  the 
span  of  control  issues,  and  (most  important)  of  their  own  strategic  plan,  ASN  (RD&A)  DASNs, 
PEOs,  and  SYSCOM  Commanders  can  construct  a  performance  measurement  plan  that 
integrates  them  with  the  program’s  BSC. 

The  table  below  shows  how  the  BSC  can  be  integrated  with  other  important  initiatives. 


ASN  RD&A 

BSC 

ASN  RD&A 
Strategy 

ASN  (RD&A)  Objectives 

PM  A/P  ART  LINK 

Warfighter 

Deliver  effective, 
affordable 
weapons  for 
warfighters 

Define  the  path  to  ForceNet  by 
collaborating  with  both  the 
operational  and  requirements 
communities  and  with  our  allies. 

-  Expanded  e- 
government 

Internal 

business 

processes 

Think  as  a 
business 

Each  DASN,  PEO,  and  PM 
should  seek  to  reduce  the 
volume  of  acquisition  documents 
by  50%,  including  only  essential, 
relevant  information. 

-  Budget  and 
performance  integration 

-  Competitive  sourcing 

Learning  and 
growth 

Integrate  systems 
and  develop 
people 

Every  DASN,  PEO,  and  PM 
should  create  a  notional 
personnel  development  plan  that 
would  identify  candidates  to 
assume  leadership 
responsibilities  at  scheduled  or 
future  transition  points. 

-  Strategic  management 
of  human  capital 

Weapon 

systems 

Run  a  tight  ship 

Every  program  will  seek  to 
continuously  cut  government 
and  industry  cost. 

-  Improved  financial 
management 

-  Expanded  e- 
government 

Implementing  Performance  Measurement  Based  on  the  BSC 

From  here,  ASN  (RD&A)  managers  can  determine  the  measures,  targets  and  initiatives 
that  will  support  their  overall  objectives.  Completing  these  three  determinations  (measures, 
targets,  and  initiatives)  will  create  a  line-of-sight  for  all  employees  in  the  unit,  provide  a 
dashboard  for  understanding  and  reporting  against  progress,  and  align  activities  with  overall 
strategic  goals. 

Figure  7  and  Figure  8  are  sample  performance  matrices  (one  each  for  a  DASN  and  a 
PEO)  including  key  areas,  measures,  weights,  and  sample  data. 
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Figure  7:  Sample  DASN  Performance 


DASN  PERFORMANCE  MATRIX  |  | 

Common  to  all  . 

Key  Area 

Measure  * 

Weight 

3  Goal 

2 

1  i 

YTD 

YE 

▼  ... 

. . 

•r  (%) 

Prior  Year 

Common  w/some  . 

. 'X 

External  Reports 

#NunrvJVIeC’urdy  breaches 

7 

0 

0 

1 

>1 

0 

0 

••^Programs  on  DAES  agenda 

6 

<3 

<4 

5 

>5 

negotiation 

| . 

ffccgCost 

%  Annual  growth 

5 

<0.4 

<0.6 

0.8 

>1 

0 

#  APB  breaches 

4 

1 

2 

3 

>3 

0 

0 

w 

Schedule 

#  APB  breaches 

4 

1 

2 

3 

>3 

0 

0 

Performance 

#  APB  breaches 

4 

<3 

<4 

4 

>4  or  KPP 

Weighting  <* . 

negotiated  . 

Program  Decision 
Meetings 

%  meetings  delayed  due  to 
documentation 

10 

<3 

3 

4 

>4 

2 

2 

1 

y  %  meetings  delayed  due  to 
unresolved  issues 

. lO**** 

. . 

1 

2 

3 

>3 

2 

3 

UJ 

$  . 

Regyiretrients 

%  ORDs  w/non-  QAI\A  chaUges 

2 

<10 

<15 

18 

>20 

4 

14 

..Ave-dayS  pending  ORD  app 

2 

<180 

<210 

285 

>285 

200 

200 

UJ . 

Ave  days  pending  APB  app 

2 

<100 

<120 

150 

>150 

115 

Measures  and  values  .•**■’’ 

. 

PPBS 

%  programs  changed  (excludes 
execution  &  taxes) 

6 

<25 

<40 

50 

>50 

0 

20 

negotiated 

%  adverse  issues  favorably 
resolved 

8 

>50 

>25 

25 

<25 

100 

30 

.  8  _ 

Fleet 

Fleet  visit  frequency 

7 

>1.5 

1 

0.8 

<.8 

2.1 

i  3  ^ 

Congress 

%  late  congressionals 

8 

<2 

<5 

5 

>5 

2 

4 

<0  o  w 

OPNAV/SECNAV 

Quality  workforce 

%  DAWIA  qualified 

4 

>80 

>70 

60 

<60 

65 

85 

LEARNING 

GROWTH 

(15%) 

%  meeting  cont  learning 
objective 

4 

>75 

>50 

20 

<20 

40 

80 

Motivated  workforce 

%  current  performance  plans  & 
scheduled  reviews 

4 

>95 

>85 

75 

<75 

75 

100 

Award-reward  rate  (%) 

3 

>15 

>10 

10 

<10 

12 

Trend 

QTR  1 

QTR  2 

QTR  3 

QTR  4 

SUMMARY  I 

*  Portfolio  weighted  average 

YTD  |  [1  |  | 

: 

3  1 

unless  otherwise  specified 

3 

Figure  8:  Sample  PEO 


PEO  PERFORMANCE  MATRIX 


Common  to  all  . 

Key  Area 

. . 

Measure  * 

Weight 

3  Goal 

2  ■  1 

YTD 

YE 

Common  w/some  . 

negotiation 

Weighting  . 

negotiated  . 

Measures  and  values  . 
negotiated 

▼ 

Prior  Year 

1 

Contract  Perf 

....©pr*' 

6 

>0.95 

>0.93 

0.92 

<.91 

0.91 

0.96 

Prog  Cost 

.,..***%  Annual  growth 

7 

<0.4 

<0.6 

0.8 

>1 

0 

0.6 

#  APB  breaches 

2 

1 

2 

3 

>3 

HH 

0 

100 

....Affordability 

%  Progs  w/goals 

4 

>80 

>70 

60 

<60 

2  -•* 

$ 

V) 

%  Progs  exceeding  goals 

6 

>10 

>5 

5 

<5 

5 

081 

0-91  1 

Schedule 

#  APB  breaches 

3 

1 

2 

3 

>3 

Performance 

#  APB  breaches 

6 

<3 

<4 

4 

>4  or  KPP 

Risk 

Risk  Index 

6 

>.9 

>.8 

0.8 

<.75 

(/) 

..r* . 

o 

o 

0£ 

CL 

Contracts 

evm  .«••• 

Current  CPAR  to  total  applicable 
^  contracts  (>$5M) 

5 

>90 

>75 

50 

<50 

65 

100 

80 

.•Cument  1  PAR  to  total  applicable 
contracts 

. . . 

**">75 

>50 

25 

<25 

30 

Ave  PALT.cjay  &■  pEfetM 2  ’mos 

applicable  contracts 

5 

5 

<180 

>85 

<200 

>60 

270 

50 

>270 

<50 

230 

65 

— 

%  Replan  IBRs  to  replans 

2 

>75 

>50 

25 

<25 

60 

80 

%  Current  EVMS  MOAs 

2 

>80 

>70 

10 

<10 

75 

Requirements 

%  ORDs  w/non-  CAIV  changes 

1 

<10 

<15 

18 

>20 

Ave  days  pending  ORD  app 

1 

<180 

<210 

285 

>285 

200 

200  | 

Ave  days  pending  APB  app 

1 

<100 

<120 

150 

>150 

0 

PPBS 

%  programs  changed  (excludes 
execution  &  taxes) 

1 

<25 

<40 

50 

>50 

20 

STAKEHOLDERS 

(20%) 

Fleet 

Miss  Cap  Rate 

5 

>90 

90 

85 

<85 

92 

Fleet  visit  frequency 

4 

>1.5 

1 

0.8 

<.8 

35 

2.1 

OPNAV/SECNAV 

Establish  Infrastructure 
plans/targets  (%programs) 

4 

>75 

>50 

25 

<25 

80 

Actual  Infrastructure 
savinqs/target  (%) 

4 

>90 

>80 

70 

<70 

90 

95 

Establish  Human  Sys  Int 
plans/targets  (%  programs) 

3 

>75 

>50 

25 

<25 

35 

80 

LEARNING  & 
GROWTH 
(15%) 

Quality  workforce 

%  DAWIA  qualified 

4 

>80 

>70 

60 

<60 

65 

85 

%  meeting  cont  learning 
objective 

4 

>75 

>50 

20 

<20 

40 

80 

Motivated  workforce 

%  current  performance  plans  & 
scheduled  reviews 

4 

>95 

>85 

75 

<75 

75 

100 

Award-reward  rate  (%) 

3 

>15 

>10 

10 

<10 

12 

Trend 

QTR  1 

QTR  2 

QTR  3 

QTR  4 

SUMMARY  1 

*  Portfolio  weighted  average 

YTD  1 

□J 

unless  otherwise  specified 

YE  I  | 
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CRITICAL  SUCCESS  FACTORS 

Several  factors  contribute  to  success  in  implementing  a  BSC  to  manage  performance. 

•  Engage  leadership:  Because  performance  management  is  a  big  commitment,  it  is 
absolutely  necessary  to  ensure  the  involvement  of  leadership  by  engaging  decision¬ 
makers  in  determining  what  activities  to  measure,  the  goals  for  each  measure,  and 
methodology  for  measurement.  It  is  essential,  too,  to  maintain  consistent  and  regular 
communications  with  leaders  to  ensure  their  continued  support. 

•  Analyze  decision-maker  needs:  End-user  involvement  is  critical  to  any  process 
redesign — particularly  for  a  performance-measurement  system  because  the  end-user  is 
upper  management.  The  metrics  put  in  place  must  truly  support  decision-makers’  needs. 

•  Establish  an  overall  metric  framework:  A  framework  is  needed  to  address  the  multiple 
metric  requirements  and  organizations  within  a  performance-management  program, 
including  issues  such  as: 

♦  alignment  with  higher-level  initiatives 
procedures,  standards,  regulations 

♦  funding 
organizational  buy-in 

•  Appoint  a  metrics  arbiter:  A  performance-management  program  creates  several 
metrics,  pulled  from  various  sources,  which  also  can  be  complicated  and  conflicting.  A 
metrics  arbiter  can  save  the  organization  time  and  trouble  by  providing  the  last  word  in 
disagreements  over  metrics. 

•  Establish  coordinating  structures:  Again,  given  the  number  and  potential  complexity 
of  metrics  measurement,  coordinating  structures  limit  confusion  and  work  by 
coordinating  metrics  collection  and  sharing  information  where  possible. 

•  Test:  A  measurement  program  is  a  big  undertaking,  and  generally  the  processes  are 
new.  In  order  to  ensure  greatest  opportunity  for  success,  it  is  best  to  run  a  limited  test  of 
the  roll-out  framework  (as  well  as  a  data  call  test),  to  evaluate  results,  improve  the 
framework,  then  roll  out  the  framework  to  the  entire  affected  community. 

•  Use  appropriate  technology:  Performance  measurement,  when  properly  implemented, 
is  often  designed  around  an  integrated  database  and  web-based  technologies. 
Information  systems  in  most  organizations  contain  incompatible  hardware,  software, 
data  structures,  and  communication  protocols,  and  these  incompatibilities  must  be 
addressed.  Additionally,  security  must  be  reviewed  and  addressed.  Transitions  to  a  new 
data  system  cannot  be  accomplished  overnight,  and  the  performance-measurement 
framework  should  be  designed  with  this  in  mind. 

COMMON  MISTAKES 

Organizations  commonly  make  many  mistakes  in  the  implementation  of  performance- 
management  programs: 

•  Metrics  don’t  relate  to  objectives/priorities:  In  some  cases,  objectives  and  priorities 
are  assumed  and  found  to  be  either  wrong  or  out-of-date  by  the  time  the  metrics 
program  is  implemented.  In  other  cases,  metrics  simply  are  not  well  matched  to  the 
organization’s  objectives.  The  danger  in  these  two  cases  is  that  metrics  become 
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irrelevant,  and  the  program’s  success  is  limited  by  a  sense  among  those  collecting 
statistics  that  the  program  is  simply  make-work. 

•  Metric  programs  take  too  long  to  produce  results:  The  assessment  and  design 
phases  of  metrics  programs  frequently  generate  an  abundance  of  new  metrics  without 
consideration  of  the  time,  complexity,  and  expense  involved  in  their  ultimate 
implementation;  this  lack  of  forethought  can  disconnect  management  expectation  from 
the  project’s  capability  to  deliver. 

•  Recycling  old  metrics:  Performance-management  programs  often  take  advantage  of 
existing  metrics,  which  can  be  valuable.  But,  there  are  considerable  risks  to  recycling 
this  data: 

*  Pre-existing  metrics  are  often  fragmented  and  do  not  form  part  of  a  cohesive  whole. 
This  is  especially  the  case  if  the  metrics  were  defined  at  a  departmental,  rather  than 
an  organizational,  level. 

*  Pre-existing  metrics  may  be  at  the  level  of  granularity,  scope,  or  focus  they  were 
when  they  were  first  conceived,  and,  therefore,  are  ill-suited  to  the  new  requirement. 

*  The  decision  to  accept  pre-existing  metrics  on  a  “temporary”  basis  until  a 
comprehensive  metrics  system  is  implemented  can  lead  to  failure.  Temporary 
solutions  tend  to  become  permanent  solutions  as  investment  dollars  are  spent  on 
implementation. 

LESSONS  LEARNED 

As  with  common  mistakes,  there  are  several  lessons  learned  among  organizations  that 
have  successfully  implemented  performance-management  programs. 

•  Leadership  must  be  committed  to  strategy  and  performance-management  program 
implementation  and  accountability. 

•  Managers  must  communicate  strategy  in  an  easily  understandable,  logical  structure  and 
framework. 

•  Organizations  must  manage  performance  with  a  deliberate  process. 

•  Alignment  and  visibility  of  all  processes  and  activities  are  important. 

•  Measurement  is  essential. 

•  It  is  important  to  select  a  good  strategic-planning  and  performance-management  model 
and  continue  to  use  it. 

•  Measurable  outcomes  are  critical  to  success. 

•  Managers  must  ensure  ownership  throughout  the  organization. 
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2003  -  2006  Sponsored  Acquisition  Research  Topics 

Acquisition  Management 

■  Software  Requirements  for  OA 

■  Managing  Services  Supply  Chain 

■  Acquiring  Combat  Capability  via  Public-Private  Partnerships  (PPPs) 

■  Knowledge  Value  Added  (KVA)  +  Real  Options  (RO)  Applied  to  Shipyard 
Planning  Processes 

■  Portfolio  Optimization  via  KVA  +  RO 

■  MOSA  Contracting  Implications 

■  Strategy  for  Defense  Acquisition  Research 

■  Spiral  Development 

■  BCA:  Contractor  vs.  Organic  Growth 
Contract  Management 

■  USAF  IT  Commodity  Council 

■  Contractors  in  21st  Century  Combat  Zone 

■  Joint  Contingency  Contracting 

■  Navy  Contract  Writing  Guide 

■  Commodity  Sourcing  Strategies 

■  Past  Performance  in  Source  Selection 

■  USMC  Contingency  Contracting 

■  Transforming  DoD  Contract  Closeout 

■  Model  for  Optimizing  Contingency  Contracting  Planning  and  Execution 
Financial  Management 

■  PPPs  and  Government  Financing 

■  Energy  Saving  Contracts/DoD  Mobile  Assets 

■  Capital  Budgeting  for  DoD 

■  Financing  DoD  Budget  via  PPPs 

■  ROI  of  Information  Warfare  Systems 

■  Acquisitions  via  leasing:  MPS  case 

■  Special  Termination  Liability  in  MDAPs 
Logistics  Management 

■  R-TOC  Aegis  Microwave  Power  Tubes 
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■  Privatization-NOSL/NAWCI 

■  Army  LOG  MOD 

■  PBL  (4) 

■  Contractors  Supporting  Military  Operations 

■  RFID  (4) 

■  Strategic  Sourcing 

■  ASDS  Product  Support  Analysis 

■  Analysis  of  LAV  Depot  Maintenance 

■  Diffusion/Variability  on  Vendor  Performance  Evaluation 

■  Optimizing  CIWS  Life  Cycle  Support  (LCS) 

Program  Management 

■  Building  Collaborative  Capacity 

■  Knowledge,  Responsibilities  and  Decision  Rights  in  MDAPs 

■  KVA  Applied  to  Aegis  and  SSDS 

■  Business  Process  Reengineering  (BPR)  for  LCS  Mission  Module 
Acquisition 

■  Terminating  Your  Own  Program 

■  Collaborative  IT  Tools  Leveraging  Competence 

A  complete  listing  and  electronic  copies  of  published  research  within  the  Acquisition 
Research  Program  are  available  on  our  website:  www.acquisitionresearch.org 
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